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ABSTRACT 

This study investigated ways of making present and 
past results of the National Assessment of Educational Progress 
(NAEP) and other data sources maximally useful as science indicators 
and to recommend possible changes in instrumentation and procedures 
that may enhance future NAEP surveys for these purposes. Primary 
emphasis was given to reviewing published and unpublished reports 
that have resulted from the national assessment and the frequent 
special data collections which have also resulted from it, as well as 
on integrating the findings into tables and text of special relevance 
to the 1S84 edition of "Science Indicators." In addition, the report 
fil<is and test score files of the National Teacher Examinations (NTE) 
were examined for similar evidence and results that may confirm or 
dxsconfirm the NAEP results. Other data sources (such as the 
Educational Testing Service Advanced Placement Program) were also 
examined. Results and recommendations are reported and discussed 
separately for national assessment, national teacher examinations, 
advanced placement scores, and for other data sources. One 
recommendation (related to NAEP) is that all scales purporting to 
measuring the same concept should be on a common scale. Among the 
findings is that the NTE data filed proved to be of limited value as 
a source of science indicators. (JN) 
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PREFACE 



The work reported here was conducted as part of a grant to Educational 
Testing Service by the National Institute of Education (NIE) for analysis of 
data fro© the files of National Assessment of Educational Progress (NAEP) and 
other sources of data relevant to the status of science students and science 
teachers in the United States. Funds for these analyses were provided by the 
National Science Foundation (NSF). The a^j^thors of this report are indebted to 
Richard M. Berry, Program Director, Stiadies and Analyses, NSF, for taking the 
initiative in requesting the work, for providing support for the work, and for 
expert cidvice to us in conducting the work. Lawrence M. Rudner and Gerald 
Kulm of NIE were helpful in managing the disbursement of the funds by NIE and 
in monitoring the preparation of project reports. 

At ETS, a number of ETS staff members assisted the authors in locating 
and processing the required data and library resources, including Ina S. 
Mullis, Associate Director of NAEP; Lucy Mitchell, Manager of Systems, SHEP; 
Edward J. Masonis, Progrejn Administrator, SHEP; and Carl H. Haag, Program 
Administrator, College Board Program Services. Albert E. Beaton served as 
technical advisor in regard to the NAEP data base. Irene Smith managed the 
preparation of materials and reports and Faith Thonpson assisted in the typing. 
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Science Indicators frc^ National Assessment and Others Sources 

Hie preparation of secondary school graduates in the area of mathematics 
and science currently is the focus of widespread concern in this country. 
Several educational commissions have pointed to lack of attainment in this 
area, and the National Science Board's Commission on Precollege Education in 
Mathematics, Science and Technology has proposed as a primary goal the 
strengthening of precollege science and mathematics education.^ In line with 
its proposal, NSF will devote a full chapter of the 1984 edition of Science 
Indicators to trends in the preparation of secondary stXKlents in mathematics 
and science. The purpose of the present study was to investigate ways of 
making the present and past results of NAEP and other data sources maximally 
useful as science indicators and to recommend possible changes in instnamen- 
tation and procedures that may enhance future NAEP surveys for these purposes. 

The primary emphasis of the work was on reviewing the large number of 
reports, both published and unpublished, that have resxilted from National 
Assessment and the frequent special data collections \Aich have resulted from 
it, as well as on integrating the findings into tables and text of special 
relevance to the forthcoming 1984 Science Indicators. 



Report of the National Science Board's Commission on Precollege Education in 
Mathematics, Science and Technology, Educating Americans for the 21st Century , 
National Science Foundation, Washington, DC, 20550, 1983, page 5. 
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Secondly, in recognition of the iitportance of conpetent instruction in 
mathematics and science at the high school level and, thus, the iaportance of 
teacher characteristics, the report files and the test score files of the 
National Teacher Examinations were examined for similar evidence and resxilts 
that may coniArm or disconfirm the NAEP results. 

Method 

National Assessment Steps 

"National Assessment has measured science knowledge and skills of 9~, 13- 
and 17-year-olds in 1969-70, 1972-73 and 1976-77." So states the NAEP report. 
Three National Assessments of Science: Changes in Achievement 1969-77 . The 
report goes on to explore the collected data on science knowledge and skills 
and to provide a context for the trends in precollege scientific achievement 
and their relative ascendency or decline among students in public and private 
schools across the nation, in addition, a more recent report, images of 
Science from the Science Assessment and Research Project of the University of 
Minnesota, summarizes the results from the 1981-82 National Assessment in 
Science. These reports are only two of more than fifty NAEP official 
publications and NAEP-related publications dealing exclusively with science. 
The number of similar publications concerned with mathematics is equally 
extensive. 

The purpose of NAEP is to provide information for governmental and 
educational policy makers. It serves as the nation's report card and has the 
responsibility to determine vrtiat young Americans know and are capable of 
doing, in addition, the authorizing legislation calls for the periodic 
reporting of data on changes in the knowledge and skills of such students over 
time. 
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The rich data base of NAEP began to be amassed in 1969 and at present 
contains records of the educational achievement of over 1,000,000 students in 
several curriculum areas, of \^ich mathematics and science are two. 

The lAEP collection of representative national data on educational 
competence in mathematics and science is an invalxxable resource in the 
development of precollege science indicators. Through matrix sampling, NftEP 
has always had the capacity for conpreh-^nsive coverage of siabject matter. In 
addition, the utilization of a deeply stratified, multistage probability 
sample design ensures that participating students are selected in such a way 
that they represent the national population of 9-, 13-, and 17-year-olds. 

The current administration of NAEP has dramatically increased the nxamber 
or background and attitude items responded to by each student. However, there 
is still a wealth of data from prior assessments \*ich further break down the 
national sample by: 

0 Region (Northeast, Southeast, Central, and West) 

o Type of community (advantaged-urban, disadvantaged-urban, and 
extreme- rural) 

0 Size of community (big city, fringes around big city, medium city, and 

small places) 
0 Grade in school 
0 Sex (males, females) 
0 Race (Blacks, Whites) 

0 Parental education (less than high school graduation, at least one 
parent \*o graduated from high school, at least one parent who had some 
post-high school education) 
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The following steps were undertaken* 

1. ihe extensive internal collection of NAEP materials were searched for 
all relevant documents, impublished papers, and compater files. 

2. Organizations and individuals who have been using the NAEP public-use 
data tapes for independent research projects were called. The purpose 
of these contacts was to identify relevant results and reports that 
may not have been disseminated to date.' 

3. A detailed search of ERIC and other conputerized files that may have 
included relevant abstracts and material was conducted. In conducting 
the literature searches, we looked not only for trend results based on 
NAEP data but also for any other trend results based on data that were 
relevant to the precollege preparation of high school students in 
science and mathematics. 

4. With the aid of ETS staff members associated with NAEP relevant 
material was identified. 

5. Limited analyses of source data were performed to obtain 
cross-tabulations and other desired descriptive statistics that were 
not in the output and reports examined. 

6. Meetings were held in Washington with NSF staff to present the 
prel5.minary results and to discuss possible additional steps, to 
interpret the results, and to discuss inconsistencies or discrepancies 
among them. 

National Teacher Examinations Steps 

In view of the critical iirjortance of competent instruction in the 
sciences and mathematics at the secondary school level, other ETS sources of 
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data on the characteristics and qualifications of science and math teachers 
were examined. An obvious first step was to conduct descriptive analyses from 
data in the NTE Programs (formerly known as the National Teacher Examinations )• 
A recent GAO report discussed how little is known about the achievement of 
math and science teachers and specifically asked; "How much can we learn from 
the National Teacher Examinations and similar state tests. ••?" Because of 
distributional limitations and confidentiality considerations explained below, 
neither NTE score nor volume data have been applied for policy purposes beyond 
the individual state level. This explor:^tory study was probably the first 
examination of the possible usefulness of NTE data in research on the 
characteristics and conpetence of teachers in the U.S. 

Uie annual volume of NTE test takers is approximately 83,000 nationally. 
Candidates, however, are not equally distributed across the United States. At 
present, about 30 states require, encourage, or offer as an alternative to 
course requirements at least some conponent of these examinations. (See 
attached chart , in appendix.) However, the nxamber of states informally 
identified as "NTE States" is much smaller. Those vAiich require the NTE Core 
Battery and/or Specialty Area Tests or use these tests as a significant 
alternative to the approved program approach are Arkansas, California, 
Louisiana, Mississippi, Ndrth Carolina, South Carolina, and Virginia, in 
addition to these. New Mexico, Tennessee, and West Virginia are major users of 
NTE Program Tests. These ten states obviously do not constitute a nationally 
representative saitple — a fact that is noted below. 

In addition to the very uneven use and distribution patterns, policy 
analysis beyond the state level has been precluded in the past because of a 
policy of confidentiality on the part of ETS. ETS will not publicly release 
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NTE test score data by state without client permission. Even volume data have 
proven difficult to interpret over time (both within states and aggregated) 
because of the frequent changes in certification options and the varying 
number of xiser states from year to year. 

Ihe NTE data include a five-year history file that contains personal 
variables for each candidate, inclxjding race, sex, age, undergraduate grade 
point average (OGPA), tmdergraduate and/or graduate major field, and of 
course, the particular NTE test or tests taken. Of the 27 Specialty Area 
Tests ( also in appendix ), data from the Mathematics Test and two science tests 
(the Chemistry, Physics, and General Science Test and the Biology and General 
Science Test) would be of most interest to the proposed study. 

An ETS statistical coordinator familiar with NTE data files and a 
professional associate examined options for "mining" the Specialty Area Test 
data for indicators of both quality and quantity of prospective teachers of 
mathematics and science. Findings of possible value regarding quantity were 
the number of candidates taking the math test and each of the science area 
tests as a percentage of those taking all Specialty Area Tests over a ten-year 
period. Coitparative distributions of these test takers by race and sex over 
five years were considered. The feasibility and value of conparing these 
volume trends by states were examined, always keeping in mind the obligation 
of confidentiality. 

Regarding the quality of prospective teachers v*o have taken NTE Tests, 
we compared the mean NTE Core Battery or Conomon Examinations scores of those 
taking the Specialty Area Tests in math and the sciences with those taking 
other Area Tests in recent years, it should be ein)hasized, however, that the 
nature of the population taking the NTE, the inportance of confidentiality. 
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the possible sensitivity of certain resxilts (e.g., racial and regional 
differences)^ and the changing character of the tests in recent years place 
limits on vise of the data. 
Other Steps 

A number of other possible data sources were examined including data from 

the: 

o ETS Secondary School Admissions Test, 
o Iowa Test of Educational Developjaent, 

o College Board Preliminary Scholastic Aptitude Test (PSAT), 

o Armed Services Vocational ^tittide Battery, 

o ETS Advanced Placement Program, 

o CEEB Achievement Tests, and 

o the High School Equivalency Test. 

The use of scores from the Scholastic Aptitude Test and the Graduate 
Record Examinations had already been considered and acted upon as part of 
previous contracts between ET-S and NSF. With one exception the possible use 
of each of the additional data sources listed above was rejected, usually 
because the sample of students taking the tests were not sufficiently 
representative of all students in the United States or, especially in the case 
of the PSAT, the results would imnecessarily duplicate the results obtained 
earlier for the SAT. 

Advanced Placement . The one exception was data from the ETS Advanced 
Placement Program. Although obtained from a small highly self-selected 
subpopulation, the results from this program were judged to be of sufficient 
interest to justify detailed examination, provided thac caution is exercised 
in interpreting the resul*:s. These results are described shortly. 
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Results 

National Assessment 

Hie primary prcxiucts of this line of work are the sections of the 1984 
Science Indicators that were prepared by Richard Berry of NSF. This 
publication will be available shortly, ihe major task as far as NAEP data 
were concerned was to update the NAEP results previously published concerning 
national trends in science and mathematics scores. 

For trends in science, the authors relied priiuarily on Three Assessments 
of Science y 1969-77 (Report No. OO-S-21) for past trend data and on images of 
Science (Hueftle, Rakow, & W^lch, 1983) for current results. TO make sure the 
later survey was designed and conducted in a way that would provide conqparable 
results, the Images of Science was reviewed by a consultant formerly 
associated with NAEP. T!he conclusion of this consultant was that the results 
of this fourth in the series of national assessments in science mot 
conventional standards in survey technology and that the resxilts coiild be 
conpared with the results of previous national surveys. His report, a copy of 
v^ich can be obtained from T. L. Hilton, noted the following: 

o More technical information in regard to the procedures used to obtain 

the data in images of Science would have been desirable, perhaps in the 

form of a technical appendix or supplement, 
o ^^ropriate adjustments were made to correct for the fact that the 

Science Assessment and Research Project (SARP) sanple included more 

larger schools than previous NAEP assessments, 
o The SARP sanple was approximately one--third the size of previous NAEP 

san^les but the absence o£ standard errors of the various siammary 

values made it difficult to assess the statistical significance of the 

results. 
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o In the absence of the actual items used in the SARP assessment it was 
difficult to evaluate their comparability to previous items. 

On balance, the consultant judged the SARP report to be of "good quality" 
and concluded that "if used with some caution [it could] serve well as the 
fourth point on the science trend line." 

Subsequently, the present authors arranged for a review of the science 
items of the SARP assessment by a subject-matter specialist at ETS vAao judged 
them to be of good quality and adequate for their purpose. 

Assembling the required NAEP data proceeded without serious problems and, 
as is evident from Mr. Berry's chapter in Science Indicators, resulted in a 
sudDstantial addition to that volume. On the basis of this work, the authors 
woxild make the following recommendations: 

1. That future reports based on NAEP data be documented in minute detail, 
probably in technical footnotes or appendices. This documentation 
should include details about exactly vdiich items were used in 
reporting summary scores and, vAaen possible, copies of the actual 
items used; details about any statistical operations performed on the 
items, and details about the sample on vAaich the descriptive 
statistics were based including standard errors. Despite the obvious 
care that has been taken in the £^st in preparing NAEP data for 
publication, the authors occasionally were uncertain about some of the 
details. 

2. That uniform procedures be adopted in regard to rounding the 
statistics reported. Some differences between statistics reported in 
one publication and, presumably, statistics based on the same data in 
other publications were attributable to different rounding procedures. 
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3. That manuscripts be proofread with xjnusual care; a few minor 
typograjiiical errors were found. 

4. That an annxial or biennial cumulative index be disseminated, including 
all reports that are available, with detailed descriptions of each 
and, preferably, a summary of the salient findings of each. This 
seems to have been done occasionally but not on a regular basis. 

5. That henceforth all scales purporting to measure the same concept be 
on a common scale. (Presumably this will now be done routinely from 
now on, through IRT equating.) 

6. That uniform procedures be adopted for reporting standard errors. The 
authors occasionally were in doiabt about vrfiether small changes were 
statistically significant. 

7. That each report routinely include information about when the items on 
which the restdts are based will be available to other researchers and 
how they should go about obtaining the items or scales. 

National Teacher Examinations 

As anticipated at the start of this study, the NTE data files proved to be 
of limited value as a source of science indicators, primarily for the reasons 
cited above. The NTE population is highly self-selected; whether a particular 
individual takes the NTE depends primarily on whether the school district or 
state in which the prospective teacher is applying for a position requires the 
NTE, and these district or state requirements change considerably from one 
year to the next, and differ markedly from one locale to another. Also, there 
are occasional special administrations of the test that contribute to change 
in the test population. Lastly, the test score files were not designed for 
the purpose of annual indicators meaning that considerable data processing is 
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necessary in order to identify subpopulations that might be reasonably stable 
from one year to the next. 

Despite these known problems the authors considered several alternative 
ways of constructing useful annual indicators. 'She first was to focus on 
school districts that were known to have required the NTE of ajplicants for 
teaching positions in the district for a nuniber of years in the recent past. 
The city of Chicago was an example. Hiis alternative was rejected since the 
nuraber proved to be too small, especially considering that our interest was 
only in the sxabsample o£ test takers \fho took the area tests in science and 
mathematics. 

A second alternative focussed on candidates taking the Area Tests of 
interest and then adjusting these scores for anrnaal fluctuations in the mean 
ability of the sample by means of the Core Battery scores, this was rejected 
because only a sKall fraction of the NTE population take both the Core Battery 
and any particular Specialty Area Test of interest. 

A third alternative, \^ich was pursued, focussed on test takers in a small 
subset of states which were known to have required the NTE consistently for at 
least the last five years. As described in the report in the appendix, these 
states were Arkansas, California, Illinois, Louisiana, Mississippi, New York, 
North Carolina, South Carolina, Tennessee, and West Virginia. C&viously, this 
alternative was not fully satisfactory since the sample of available states 
over represents states in the southern part of the United States. However, 
change in the mean scores of the test takers in these states was judged to be 
of interest even though the mean scores for these states were not necessarily 
representative of all the states. 
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As described in the report in the appendix, the mean scores on the Area 
Tests related to science and mathematics did change from 1979 through 1984 in 
nonrandom ways* However, there was no consistent pattern in the changes and 
the changes were not large. These results could be viewed as encouraging in 
that there was no evidence of an expected marked decline in the test scores of 
prospective math and science teachers. But in view of the possible bias in 
the results, the authors' advice is that the results be interpreted with 
caution, tto recommendation was made that the NTE test scores be used as 
science indicators. 
Advanced Placement Scores 

As mentioned above, the third possible source of science indicators that 
was examined with some care was the data files of the Advanced Placement (AP) 
Program. AP courses and examinations are given at over 20% of American 
secondary schools to 15% to 20% of their most able college-botind students. 
Participants are by definition doing college level work and are high achievers 
cjid generally highly motivated. Each examination (with the exception of 
Studio Art) includes both an objective, multiple choice section and a free 
response or essay portion. In each subject area, a groiip of teachers grades 
the free reponse part of the examination. Uiese teachers, from participating 
schools and colleges across the country, are organized and directed in their 
grading by a chief reader, ^o typically is a college professor. Final 
grades, based on the student's entire examinations (with free response and 
multiple choice questions appropriately weighted) , are reported on a 5 point 
scale: 5- extremely wsll qualified, 4- well qualified, 3- qualified, 2- 
possibly qualified, 1- no recommendation. Participating colleges normally 
honor grades of 3 or higher. 
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As described in the report of this work, which is inclxided in the 
appendix, changes in the mean sumroary scores fran one year to the next do not 
necessarily reflect reliable changes. Itie grade data are based on scores from 
both the multiple choice portion and the free response parts of the 
examinations but only the multiple choice portion is equated ftm one year to 
the next. Furthermore, cut scores vary from year to year for each 
examination, reflecting changes in levels of exam difficulty, and the chief 
readers often attempt to maintain similar percentages of students receiving 
each of the scores from 1 to 5, froQ year to year. Hius, changes in mean 
scores during the last ten years were jixlged to be unsuitable as indicators of 
changes in the achievement of students taking the AP examinations. 

Examination of the nxjonber of students taking the AP exams over the last 
ten years indicated that interest in biology, chemi:»try, and physics — relative 
to other subjects — remained very much at the same level over the last ten 
years v^ile relative interest in mathematics declined dramatically. Hiese 
volumes, however, may have been affected by the availability of teachers in 
mathematics and science and, since there is no feasible way of ascertaining 
the extent to vihich this factor may have influenced volumes, the authors' 
recommendation was that these data be used as science indicators with caution. 
Other Data Sources 

Of a number of other possible data sources examined, one was considered 
promising. This is the nimiber of math and science courses that secondary 
students enroll in during their last three years of high school attendance. 
To investigate the validity of this indicator, the preliminary results of the 
study of Excel?,ence in High School Education, then underway at ETS, were 
examined. This study was based on longitudinal data for the High School and 
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Beyond subjects vho were surveyed as sophcmores in 1980 and as seniors in 
1982. Identical short tests of achievement in mathematics and science were 
given to the approximately 23,000 students vitio participated at both times. 
Transcripts of the hi^ school academic work were obtained for one-half of the 
subjects after they were graduated. From the transcripts, counts were 
obtained of the total number of mathematics courses and the total number of 
science courses in which each student had enrolled. Two outcome measures were 
of interest: the test scores of the students in their senior year, and the 
residual gain in test scores from the sophomore to the senior year. (The 
latter measure is the gain in test score adjusted for difference among the 
sxibjects as sophomores.) 

Hie product moment correlations between the number of courses enrolled in 
and the two outcc^oe measures were as follows: 

Senior Residual 
test score gain 

No. of math courses .55 .29 

No. of science courses .49 .23 

These correlations — for a large national saitple of high school students — 
point to a strong relationship between enrollment in high school math and 
science courses and the measured acliievement of the subjects. The authors 
concluded that these correlations, along with similar findings in the 
research literature, were sufficient evidence of the validity of the number of 
science and math courses as indicators of probable achievement in these areas 
during high school, and recommended that the statistics be given serious 
consideration as science indicators even though data on course enrollments are 
available only from quite widely spaced studies such as the ETS Growth Study 
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(1963-69), the 1972 National Longitudinal Study (1972 to present), the 
Department of Labor National Longitudinal Surveys (1979 to present), and HS&B 
(1980 to present). 
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NTE PROGRAMS 
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Area Tests 
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Y 
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X 
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Nev York 
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Y 








North Carolina . 


0 


Y 


Y 






y 


North Dakota 














Ohio 






Y 








Oklahoma 














Oregon 






Y 








Pennsylvania 


0 


0 










Rhode Island 






Y 








South Carolina 




Y 








y 


South Dakota 














Tennessee... 


Y 




Y 








Texas 








y 






Utah 






Y 








Vermont 














Virginia 


Y 


Y 










Washington 














West Virginia 


X 


X 










Wisconsin 














Wyoming 














BEST copy AVAILABLE 

X Alternative to course requirements 
/ab Z Completed study, currently under considsration 
CD?r>*/6/83 for 1984 
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APPENDIX B 

Means and Standard Deviations of Scores on 
the Common Examinations, and Correlations 
(r ) Between Avea and Common Examinations 
Scores for Examinees with Composite Scores 
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SupplftMnt to Uafltc Bl» Inttrpretins Matloiuil TMchtr Exaaiiuitloiic Scores 

lS!Sill. ^U^(!^\T/^^^^^ ^"^"^ ^* ^^ticcB. and CorrolotloM (r,^) Botwta Art. «nd Co»on BKamnatloo. Score. «r 



Aroe ExaBlMtioo 



ExMlneee vith Uee Tb«a HMterU 

Art Education 
— ' Biology and GMorol Sdonco 

Buoineeo Education 
*-^7ChMl4try» Phyeica. and Ganarel Sdmica 
Early Childhood Education 
Edur^tion in the Eleaantary School 
Ed'tcatioo of tha Mentally Batardad 
Engli^s Unguaga and Utoratura 
French 

Hcmm Ecoooaica Education 
Induatrial Arte Education 
Introduction to tha Taadiing of Kaading 

— - Mathenatica 

TCua iTYdiicatton 



Fhyalcal Education 
Social StudlM 
Spanish 

Spaach<*CoHniiication and Thontra 

EiuMBlneaa trith NaaterU or Doctorate 

Educational Adniniatration and Suparviaion 
Guidance Coucaalor 

Media Spacialiat— Library and ArV Servicaa 
Beading Specialiet 
Speech Pathology 



Nuid»er of 



2,924 



TTT 
519 
M,9« 
a7,638 
7»742 
5,3U 
504 
2»651 
1,242 
325 
3,052 



4,UB 

9,378 
6,578 
1,231 
761 



Frofeaaional 

Education 



Written 
Englieh 
Eicpreeaion 



Social Studiea, 
Literature, and 
the Pine Arte 



Science and 
MatbeMtica 



Weighted CoMMn 
Exanlnationa 
Total 



Mean S,P, t^f Mean S,D, r^ Mean S.O. r^ Mean 8.0. r^ Mean S.D. rxy 



55.2 
57.4 

ir 



60.2 



9.8 
10.6 

TXT. 



3571" 

55.6 

56.9 

59.8 

59.9 

54.0 

49.3 

59.8 

58.4 



35:1- 

50.0 

56.3 

52.5 

56.0 



3" 
10.4 

11.2 
10.9 
10.2 
9.4 
10.8 
11.3 
11.6 
10.6 



777" 
10.5 
U.O 
U.3 
10.1 



.726 
.778 
.852 
.735 
"7875" 
•878 
•858 
•802 
•469 
.874 
•808 
•874 
.661 



56.2 
58.0 



60.8 
5^.5 
55.1 
54.9 
64.3 
65.0 
54.9 
49.3 
59.8 
60.1 



10.4 
10.6 

9.8 



.618 
.655 

"TTST 
.632 



10.8 
U.O 
10«5 
10.0 
9.5 
10.5 
10.3 
11.3 
10.5 



59.3 
57.8 
49.6 
61.1 



• 723 
.838 

• 794 
.193 
.800 



59.1 
50.4 
56.7 
53.7 
58.4 



9.9 
10.8 

13.1 
10.2 



.740 
.751 
.703 
.754 
.476 
.742 
.682 
.739 
.603 



9.5 
10.2 

9.8 



.784 
.758 



"7725" 
.715 



57.1 
66.2 



"52T7" 
72.4 



9.3 
10.7 



.697 
.670 
.106 
.694 



874 


62^5 


11.0 


.872 


58.1 


11.2 


.718 


351 


61.C 


10.7 


•822 


59.9 


11.1 


.719 


239 


63.0 


10.9 


.818 


64.0 


11.0 


.702 


258 


66.5 


10.0 


.845 


63.4 


10.6 


.730 


281 


63.8 


8.2 


.653 


63.9 


9.1 


.563 



53.1 
54.1 
54^0 
63.0 
63.2 
52.4 
51.8 
60^5 
57.2 
58.2 
50.2 
60.8 
54.4 
59.2 



60.2 
60.2 
66.4 
64.5 
61.2 



m.2 
10.4 
10.0 
10.3 
9.6 
10.0 
10.8 
12.1 
10.4 



8.9 
9.0 
U.l 
11.5 
10.2 



11.9 
11.1 
11.2 
11.0 
8.8 



.742 
.737 
.707 
•851 
.511 
.761 
.728 
.722 
.599 

TtoT 

.702 
.831 
.196 
.776 



.744 
.757 
.738 
.712 
.493 



TO" 
10.1 



.648 
.849 



TtST 
.801 



568 86 
598 94 



TIT 
635 



53.6 
55.2 
54^8 
58.7 
60.5 
5o.l 
58.3 
57.7 
66.7 



91 



.874 
.823^ 



59.8 
55.0 
57.7 
53.3 
56.7 



59.2 
58.2 
59.3 
58.9 
61.0 



10.1 
10.7 
10.2 
10.2 
9.0 
10^2 
11.9 
11.8 
10.4 

TT 
10.0 

10^7 
10.7 
10.1 



11.1 
10.1 
10.7 
10.6 
8.9 



.746 

• 773 
.715 

• 731 

• 360 

• 773 
•840 
•727 
•708 



^68r 

• 743 

• 719 
•160 
.709 



.681 
•680 
.638 
•686 
.588 



542 
551 
555 
608 
614 

543 95 
522 103 
595 107 
604 94 



"578 W 

514 88 

578 98 

533 105 

572 SO 



607 100 

604 97 

630 97 

638 94 

625 73 




^795 
.867 
.859 
.191 
.856 



.876 
.836 
.834 
.846 
.689 
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Sttppl«M&t CO lMfl«t Dlt Iiic«rpr«cliig ItacioiMl tMelior gsMliutlon* Scorat 
TAILI 4t MMiit and Standard DavUtlooa of Seoraa oo tba Coanoo txaainacloiiai aod Corralatlona 
licaal naaa «fitti Coa^alca 8coraa* 



(r^y) Batvaan Araa and Ccmon EiranlAaciona Seoraa for 



Araa Examination 



Bxattiaaaa mtli Uaa thaa HiatarU 

Art Idttcation 

Biology and Ganaral Scianca 
f'^xsaaa Education 

Chaalatryi rbyaica, and Caoaral Scianca 
Early Childhood Education 
Education in tha Elanantary School 
Education of tha Mantally latardad 
EnglilUi Uoguaga and Litaratura 
Franch 

HoM EconoKica Education 

Induatrial Arta Education 

Introduction to eha Taachiag of Eaadina 

HathaMtica 

Huaic Education 

rhyaical Education 

SocUl Studiai 

Spaoiah 

Spaach^C n — ii n ication and lhaatra 

KKito aaa vlth MaatarU ar Doctorata 

Educational Adnlniatration and Supawlaioa 
Cuidanca Counaalor ' 
Madia Spaciallat-Lihrary and A-V Sarvicaaj 
taadicf Spacialiac 
Spaach Pathology 




2,274 
2,671 
3,240 

466 
15,76S 
24,613 
7,428 
4,589 

442 
2,230 
1,084 

386 
2,548 
4,150 
9,460 
5,577 

733 

616 



918 
388 

203 
372 
263 



Frofaaalonal 

Education 



Urittan 
Engliah 
Expraaaion 



Social Studiaa, 
Litaratura, aod 
tha Finn Arta 
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Naan 8.D. 



Haaa S.D. r 



2QL 



54.9 
56.0 
51*3 
60.5 
55.5 
55.8 
56.7 
60.4 
59.9 
54.2 
49.7 
56.7 
56.3 
55.8 
50.2 
56.7 
54.8 
56.0 



61.8 
61.2 
61.9 
66.2 
63.5 



10.2 
10.3 
10.6 
10.1 
11.0 
11.1 
11.0 

9.9 
10.0 
10.7 
11.6 
11.7 
10.4 

9.8 
10.6 
10.7 
11.5 

9.9 



10.4 
10.3 
11.0 
9.2 

7.a 



.717 

• 750 
•849 
•656 
•875 
•872 
.644 
.761 
.512 
•857 
.766 
.668 
.614 
.691 
.836 

• 768 
.193 

• 797 



.650 
.820 
.647 
•822 
.585 



55.7 
58.3 
54.6 
60^9 
54*4 
54.8 
54^3 
64.4 
65.6 
54^3 
49.2 
58^4 
60.0 
56.9 
49.8 
56^8 
57.0 
56.5 



56.9 
56.9 
64.3 
63.1 
63.3 



10.4 
10^5 
10.4 
10.8 

10.8 
10.9 
10.6 

9.9 
10.1 
10.6 
10.3 
11.6 
10.5 
10.2 

9.8 
10.9 
13.4 
10.4 



10.6 
11.5 
11.8 
10.3 
9.3 



•649 
.665 
.742 
.605 

• 740 

• 745 
•696 
.743 
•496 
.727 
•661 
.744 
•587 
•641 
.665 
•668 
.109 
.712 



.690 
.665 
.770 
.726 
• 547 



58.5 
58.0 
49^2 
61.4 
52.4 
53^5 
52^7 
63^7 
63^3 
5U6 
51.6 
57^5 
57.1 
58^1 
49.3 
61*4 
56.9 
58.4 



58*8 
59.9 
66^1 
64.1 
61.5 



10.0 
^0.4 
9.8 

10,3 
If/. 4 

ia.7 

10^3 
10.4 
10.5 
10^0 
11.0 
12.6 
10.5 
9.6 
9.3 
11^2 
12.1 
10.5 



11.4 
11.7 
11.7 
10.5 
10.1 



.778 

.750 
.735 
•672 
.733 

• 732 
.703 
.645 
•551 

• 740 
.739 

• 731 
•562 

• 708 
.694 
•834 
.218 
.754 



• 723 
.735 
.758 
.672 
.466 



Scianca and 
Mathamatica 



Naan S^D. r 



^^.4 
52.2 
70.3 
53.2 
54.6 
53^8 
58.8 
59^9 
55^\ 
57^6 
56^8 
65.3 
58.6 
54.1 
57.4 
54.2 
55.9 



58.5 
56.9 
59.9 
59.7 
60.1 



9»8 

10.1 
10.1 
10^0 
10.2 
10^7 
10^4 
10^1 

9.6 
10^1 
11^8 
11.4 

9^9 

9^9 
10^0 
10.5 
11.3 

9^7 



10.8 
10.6 
11.0 
9^5 
8^5 



•^2 
.631 

• 750 
.790 

• 761 

• 775 

• 728 

• 714 
.486 

• 772 

• 837 

• 739 
•472 
.6M 

• 746 
.737 
.157 
.HI 



.675 
• 690 
•692 
.620 
.569* 



Uftightad 
Exaalnatiooa 
Total 



Haan S.D^ r 



xy 



562 
599 
514 
632 
541 
549 



90 
91 
92 
91 
96 
99 



548 96 

612 90 



614 
538 



90 
93 



521 104 

579 109 

600 92 

574 87 

509 89 

581 97 

554 108 

568 89 



597 95 

596 98 

627 102 

638 85 

621 74 



.799 
.855 
.876 

• 769 
•883 
.883 

• 848 
.862 
.572 
•883 

• 850 

• 861 
.684 
.782 
.865 
.870 

• 195 
.850 



.831 
• 856 
.826 
.432 
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APPENDIX C 



Memorandum by Barbara Pitcher on NTE 
Programs Data for NSF, September 11, 1984 
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Memorandum for: TOh HILTON 



cc: Robert Altman 
Penelope Engel 
Jane Faggen 
Marlene Goodison 
Catharine Havrllesky 
Alice Irby 
Ed Masonls 
Carolyn Naaaad 
Craig Mills 
Lucy Mitchell 
Nancy Petersen 
Dawn Robinson 
Janice Scheunenan 
Billie Slaughter 
Frances Swineford 



Subject: NTE Programs Data 
for NSF 



Date: September 11, 1984 

From: Barbara Pitcher '^^^ 
20-P, Ext. 5967 



Reference: My memorandum to Lucy Mitchell, 
on OTE Programs Data for NSF, 
issued August 15, 1984 

The work described in the referenced memorandum has been completed and we 
have used the resulting tapes to run counts, select samples and obtain means 
and standard deviations as requested* 

Enclosed are tables showing the numbers of examinees, means and standard 
deviations for all nine of the tests we used* They are: 

Biology and General Science (03 BGS) 

Chemistry, Physics and General Science (07 CPS) 

English LansMge and Literature (04 ELL) 

Mathematics (06 MAT) 

Social Studies (08 SS) 

Business Education (10 BE) 

Early Childhood Education (02 ECE) 

Education in the Elementary School (01 EES) 

Physical Education (09 PE) 



NSF was primarily interested in science and mathematics tests (BGS, CPS and 
MAT), English (ELL) and Social Studies (SS) were added for contrast in other 
subject areas. The other four tests were also included in the computer run 
because they are relatively large-volume tests of particular interest to NTE 
Programs staff. There are 16 additional Specialty Area tests. 
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31 



Prna™?"*??*^* ^^^''"Pl^s "«ed for this analysis were selected from NTE 
Progranu; files, a. described In the referenced memorandum. That Is^a year 
TJ;8'?::%h"1/° "° ''^^ J^^June (except for 1978-79 since NoveibeJ H 
1978 is the oldest test date currently on the history file). 2 Examinees with 
multiple scores for the same f.t within a defined year were repj^nt" 
the first «core within that year. Repeater .ceres Lro.n««'w«e included 
in the -PProprlate years. For example, suppose a person tJok the same te^t In 

?Sirners^ • JT"^ ^T^"^"" ^^^l ^nd Aprirj982 

fSlloSs: represented three times in the extracted flies, as 

1979- 80 November 1979 (NOT February 1980) 

1980- 81 November 1980 

1981- 82 November 1981 (NOT April 1982) 

The samples were further restricted to include examinees testt 1 at centers in 
to*flS^^Sf certification requirements across tU* years 1978-79 

to 1983-84. These states, specified for us by NTE Program Direction (Ed 
mlTiil] *r Arkansas, California, Illinois, Louisiana, 

Mississippi New York, North Carolina, South Carolina, Tennessee and West 

°^ Illinois and New York, the numbers of examinees were 
consistently small, since only certain school districts in Chicago and nL 
York City required the tests - not the entire states. 

Testings from national and special administrations are Included. The 
A^^iZ lt shows the dates of the national administrations that occurred 

tf^^L J^"«-P"^?*^ f^,^""'^ ""'^y- <=he first year, 

defined as November 1, 1978 - June 30, 1979 Includes no summer national 
administration and that the third year, July 1, 1980 - Jur30, fSsrincludes 
two summer national administrations (July 19, 1980 and June 20, 1981). Since 
the first year was not a full year, it was dropped from the stidy. 

-^^£51 . . National Test Dates 



78- 79 Nov. 11, 1978 Feb. 17, 1979 

79- 80 July 21, 1979 Nov. 10, J979 Feb. 16, 1980 

80- 81 July 19, 1980 Nov. 8, 1980 Feb. 21, 1981 June 20,1981 
I; I; Nov. U, 1981 Feb. 20, 1982 April 17, 1982 

ICll ^^^^ April 30, 1983 

83-84 Hov. 12, 1983 April 14, 1984 



2 

Since the first year, 1978-79, included no summer administration and 
essentially no special administrations, it was decided to drop it from the 
through II Jr^"*^^ "''^ ^"^^ covering a full year, from July 1 
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Info^mfJlin^ ^^"^^""^ dlstrlbutloos of examinees by background 

f^Jror.;""'/"" educ-tlonal level, are also Included^orthe 

" ^° questions than they answer. Why, for exaaole is 

Meut'^'rSatf: 1 m";'?"\'" c-tegory/-Natlve A-erlian! EsS o^ 
Aleut relatively hl«h In the first three years and low In the last tJS (A 

difrerences that showed up In these tables?) 

five^Slts^SI' ctI °^L"i:r °f the first 

not o IS^^^^^^^^ r -"-^^^^^^^ r^siLToinTLtLo-^ ^pJes 

very'Lut!;"s'abLt''*5^lJ!i^'*°"' ?<^<^Itlonal data and/or analyses, one should be 
thl dSr tL!o ^ conclusions about trends one thinks one sees In 

the data. There are numerous factors that could Influence these datr o- 

SI TuLiz th«.ii°r» TV""'- »-ppi"»"d.^ror 

iof^o ^vf ^^^^"^^ ^"""^ "°te that the examinees who took these 

TrTlJ'tr P"^"'"*^^^ -PPlyi"8 for teacher certification; the? Slfnot neces- 
sarily become teachers), the amount and recency of preparation In cS^rsework 
related to the subject area. A serious limitation of ?he daS In S?S 

alZ;:. '^'^'^ °^ information about when a person's twining was 

acquired, for instance. Some may be currently undergoing their tr^^nJnS? 

Tll^Zl lZ\'^st; — ^ between^cSiJmg tLTJSning 
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Test 
03 BGS 



07 CPS 



04 ELL 



06 MAT 



08 SS 



Year 

79- 80 

80- 81 

81- 82 

82- 83 

83- 84 

79- 80 

80- 81 

81- 82 

82- 83 

83- 84 

79- 80 

80- 81 

81- 82 

82- 83 

83- 84 

79- 80 

80- 81 

81- 82 

82- 83 

83- 84 

79- 80 

80- 81 

81- 82 

82- 83 

83- 84 



Ns 

1,144 
1,545 
1,048 
1,013 
1,211 

223 
343 
238 
249 
327 

2,144 
2,668 
1,843 
1,550 
1,522 

1,298 
1,820 
1,359 
1,449 
1,933 

2,480 
3,237 
2,106 
1,736 
1,852 



Mean SD 



618 
604 
618 
605 
619 

572 
563 
569 
580 
584 

575 
574 
584 
581 
584 

566 
562 
563 
565 
570 

565 
561 
570 
565 
571 



89 
89 
87 
87 
84 

93 
84 
93 
90 
96 

91 
91 
94 
94 
96 

89 
89 
83 
80 
81 

91 
93 
93 
92 
9? 




02 ECE 



01 EES 



09 PE 



Year 


Ns 




oU 


79-80 


1,040 


584 


85 


80-81 


1,275 


588 


85 


81-82 


846 


578 


87 


82-83 


629 


580 


84 


83-84 


ww7 


joy 


OH 


79-80 


5,425 


580 


106 


80-81 


5,787 


568 


108 


81-82 


3,836 


574 


103 


82-83 


3,170 


559 


106 


83-8/1 


3,422 


566 


106 


79-80 


8,890 


582 


97 


80-81 


11,570 


582 


98 


81-82 


9,370 


581 


93 


82-83 


8^083 


577 


90 


83-84 


8,628 


578 


88 


79-80 


3,109 


589 


85 


80-81 


3,611 


588 


87 


81-82 


2,635 


592 


81 


82-83 


2,2' 


590 


84 


83-84 


2,10:^ 


594 


81 
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Advanced Placement Analyses 
TABLE I. AND CHART I. 

SUMMARY; Advanced Placement (AP) courses and examinations are given at over 

20% of American secondary schools to 15 to 20% of their college-bound 
students. Participants are by definition doing college-level work and 
are high achievers and generally very highly motivated. Table l.^ 
attached, shows the numbers of examinations given in eiv ht subject 
areas from 1975-1984. Chart I., also attached, graphically depicts 
the changes. These data provide some indication of the importance of 
the various subject areas to high-achieving students over the last 
ten years. (Table I. and Chart I. should be considered in conjunction 
with Table II., which indicates the relative importance of each 
subject to all others each year for the last fen years.) 

Table le and Chart I. indicate that the volumes of all eight ap 
examination subject areas shown have increased over the ten-year 
period. Those with the highest volumes (American History and English) 
increased the most in terms of both absolute numbers and rates of 
increase (211% and 177% respectively). (Chemistry experienced the 
third largest rate increase (176%), although it remains a relatively 
lov volume program. The other sciences (Physics and Biology) had 
relatively low rates of increase (162 % and 136% respectively), and 
Mathematics (Calculus), currently the third largest in volume, had the 
next to smallest rate of increase (134%). 

Taking an AP course usually suggests students' particular interest in a 
subject area and their intention to pursue that subject as a major 
field of study in college. However their interest is likely enhanced 
by the fact that advanced standing is given by many colleges and 
universities to those who score well. Since some students take more 
than one AP course, all test-takers are not necessarily future majors 
in the subject of each test. 

CAVEATS ? The AP exams included in Table I. and Chart I. were selected to 

encompass the math and all the science exams as well as several other 
exams for comparative purposes. These exams were relatively stable 
programatically throughout the ten year period. That is, no major 
changes took place in these programs which would have caused large and 
lasting volume changes. A possible exception is English, which expanded 
in 1980 from English Literature and Composition to include an English 
Language and Composition exam as well. A significant volume increase 
occurred at that time, although the rate of increase declined in 
subsequent years. 

In addition to students' interest in a subject area, volumes maybe 
affected by the availability of teachers to teach the courses. There 
has been some speculation that the small increases in the math and 
particularly the science volumes may be due to a relative scarcity 
of available teachers. ^ 
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TABLE 1. 



ADVANCED PLACEMENT EXAMINATION VOLUMES 
in Selected Subject Areas 
1975-1984 





1975 


1976 


1977 


1978 


1979 


1980 


19Pi 


1982 


1983 


1984 


% inc. 
























•75- '84 


Am. History 


16,068 


18,718 


21,325 


24,444 


28,222 


32,098 


35,999 


38,286 


43,844 


49,939 


211 


Biology 


8,206 


9,482 


10-530 


11,342 


12,835 


13,549 


15,199 


15,947 


17,804 


19,387 


136 


Chemistcy 


4,181 


5,341 


5,559 


6,270 


7,016 


8,209 


8,877 


9,476 


10,291 


1! .539 


176 


English 


25,656 


29,503 


32,142 


36,334 


41 ,975 


49,125 


55,010 


58,591 


65,260 


71 ,263 


177 


French Lang. 


3,029 


3,374 


3,601 


3,994 


4,409 


4,320 


5,352 


5,782 


6,129 


6,843 


126 


LatinAetgil 


624 


745 


841 


880 


1,016 


1,122 


1,261 


1,311 


1,529 


1,704 


173 


Math (Calc.) 


17,090 


19,065 


20,317 


22,510 


24,727 


27,879 


30,558 


31 ,918 


35,489 


39,962 


134 


Physics 


3,200 


3,663 


4,196 


4,556 


5,039 


6,222 


6,481 


6,804 


7,376 


3,3S0 


162 



Sgu^: -Advanced Placement Examination Volume Changes.- (Table prepared by Educational Testing Service, Princeton, 
Princeton, New Jersey, 1984, for the Advanced Placement Program of the College Board.) 
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CHART I. 

ADVANCED PLACEMENT EXAMINATION VOLUMES 
in Selected Subject Areas, 1975-1984 



BEST COPY AVAILABLE 




f r 



J a t — i — a 



e n c h 



1976 



1977 



1978 



!979 



1980 



1981 



1982 



1983 



1984 



: "Advanced Placement Examination Volume Changes." (Table prepared by Educational Testing 

Service for the Advanced Placen-ent Prooram of the Colleoe Rnard. Prinrpton. N .1 . iq.q4 
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TABLE IZ. 



SUMMARY : Advanced Placei^ent (AP) courses and examinations are given at over 20% 
of American secondary schools to 15 to 20% of their college-bound 
students. Participants are by definition d^ing college-level work and 
are high achievers and generally very highly motivated* The attached 
chart shows the percentage of all of the AP examinations that were 
given in each of 13 svibjects from 1975-1984. This thus gives iji 
indication of the relative importance to high-achieving students of 
the various subjects over a ten-year period. Students' interest in 
taking AP courses is enhanced by the fact that advanced standing is 
given by many colleges and universities to those score well. 
Taking an AP course usually suggests students' particular interest in 
a subject area and their intention to pursue that subject as a major 
field of study in college. Since some students take more than one AP 
course, hbwever, all test-takers are not future majors in the subject 
of each test. 

The attached table shows that, as indicated by AP exam volumes, 
student interest in biology, chemistry, and physics, relative to other 
subjects, has remained very much the same over the last ten years, 
while their relative interest in mathematics has declined dramatically, 
more than for any other subject. Math (Calculus), in fact^ dropped 
from subject of second greatest interest (English being first) in 1976 
to third, surpassed also by American History, in 1977 and thereafter. 
Biology, Chemistry, and Physics were in 4th, 6th, and 8th place in 
1975 and 1984. Unfortunately trend data are not available for AP 
Computer Science, the exam for which was administered for the first 
time in the Spring of 1984. 



CAVEATS ; Certain temporary fluctuations in AP volumes may be partially explained 
when specific programs have been added or dropped. Ebr example: a 
third Art examination (in Drawing) was added in 1980 and the Art total 
increased slightly from 1.0 to 1.1%. Also in 1980 an English Language 
and Composition exam was added. A slight decrease in the Literature 
and Composition volume occurred, but Total English increased one per- 
centage point. The German program added a Language exam in 1980 (in 
addition to Literature), and the Total German percent increased by .3 
points to .8% of all AP test-takers. Total German dropped dDwn to .7% 
in 1983 at the same time that German Literature was discontinued. In 
addition, increases were noted in Music Total in 1978 (from .3% to 
.7%) at the time the Music Theory exam was added, and in Spanish Total 
in 1977 (from 1.8% to 2.8%) when the Spanish Language exam was included 
(along with Spanish Literature) . 

These possible program-induced fluctuations, however, should not be 
strong enough to explain a steady trend over a ten-year period within 
a subject area. 
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BEST COPY AVAILABLE 

ADVANCED PLACBKBNT EXAMINATIONS TAKH? 1975-1984 
PERCENT OP TOTAL BY SUBJECT 





• 75 


'76 


1 77 


/O 




ion 


1 A 4 

'81 


*82 


'83 


'84 




ft 

« 


ft 


ft 






% 


% 


% 


% 


% 


American History 


18.7 


18*9 


1 Q 


1 Q Q 




on n 
zU • U 


^n o 
20*2 


20 .3 


20.8 


20.8 


*Art 


1.0 


1.0 


9 


1 n 




I • 1 


1 •! 


1.2 


1.1 


1.1 


Biology 


9.7 


9.6 


9^7 


Q 

7 • O 




ft R 


o c 


8.4 


8.4 


8.1 


Chemistry 


4.9 


5.4 


5.1 


5.1 


5.0 


5.1 


5.0 


5.0 


4.9 


4.8 


♦English 


29.9 


29.8 


29.5 


29.6 


30.1 


30.6 


30.9 


31.0 


30.9 


29.7 


European History 


5.1 


5.3 


4.9 


4.8 


5.0 


5.1 


5.2 


5.3 


5.5 


5.3 


French 


3.5 


3.4 


3.3 


3.3 


3.2 


3.1 


3.0 


3.1 


2.9 


2.9 




• 7 


• 6 


.6 


.6 


.5 


.8 


•8 


.8 


.7 


.7 


Latin 


.7 


•8 


.8 


.7 


.7 


.7 


.7 


.7 


.7 


.7 


Math 


19.9 


19.3 


18.7 


Ifl i 


17 7 


17 A 


17 O 
1 / .Z 


16.9 


16.8 


16.7 


♦Music 


• 4 


• 4 


.3 


.7 


.7 


.6 


•4 


.3 


.3 


.3 


Ph vrIcs 

^ ti yo Aw o 


.3 • / 


^ • / 


3*8 


3,7 


3.6 


3.8 


3.6 


3.6 


3.5 


3.5 


♦Spanish 


1.8 


K8 


2.8 


2.9 


3.1 


3.2 


J • 4 




3. 5 


O A 

3.8 


Total Percent 


100.0% 


100.0% 


100.0% 


100.0% 


100.0% 


100.0% 


100.0% 


100.0% 


100.0% 


98.4%** 


Total Number of 






















Exsuninations Taken 


85,786 


98,898 


108,870 


122,561 


139,544 


160,214 


178,159 


188,933 


211,160 


239,666 


* See Caveats section 


attached. 




















** Total for 1984 does 


not equal 


100% because Computer 


Science 


(not shown) was 


included 


for the fi 


rst time. 





SOURCE; -Advanced Placement Examination Volume Changes." (Table prepared by Educational Testing Service for the 
Advanced Placement Program of the College Board. Princeton, New Jersey, 1984.) 
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TABLE III. 



SUMMARY: 



Advanced Placement (AP) courses and examinations are given at over 20 percent 
of American secondary schools to 15-20 percent of their college bound students. 
Participants are by definition doing college-level work and are high achievers 
and generally very highly motivated. Each examination (with the exception 
of Studio Art) includes both an objecwive (multiple choice) section and a 
free response or essay portion. In each subject area a group of teachers 
grades the free reponse part of the examination. These teachers, from 
participating schools and colleges across the country, are organised and 
directed in their grading by a chief reader, who is a college professor. 
Pinal grades, based on students* entire examinations (with free response and 
multiple choice questions appropriately weighted) , are reported on a 5-point 
scale: 5-extremely well qualified, 4-^ell qualified, 3-qualified, 2-possibly 
qualified, 1-no recommendation. Participating colleges normally honor grades 
of 3 or higher. 



The attached table shows the mean grades in all subject areas from 1975-1984. 
no interpretation of these scores is provided here, as BTS d>es not recoMend 
that these data be used for Science Indicators or for any other trend analysis 
purpose. This table is being provided for information purposes only, as NSF 
requested an update on the Lyle Jones article of 1981, (See below.) 

CAVEATS: 



ETS does not recommend the use of AP mean grade data for trend analyses. 
The grade data are based on scores from both the multiple choice portion and 
free response parts of the examinations, fiowever, only the multiple choice 
portion has been equated. The scores from both portions of the exam are 
weighted and combined and placed on scales that range from a minimum of 0-9 for 
the Mathematics and Physics C exams to a maximum of 0-200 for Spanish Literature, 
History of Art, and the Music exams. Cut scores are established at four 
different points along these scales to designate a grade of 1 , 2, 3, 4, or 5. 
Cut scores frequently vary from year to year for each examination, reflecting 
changes in levels of exan difficulty, and they also differ across examinations. 
In addition, the chief readers often attempt to maintain similar percentages of 
students receiving each of the scores from 1-5 from year to year, unless there 
is a reason not to. 



For all these reasons, when considered in conjunction with the fact that mean 
grades are not equated from year to year, it is clear that the use of AP 
grades for trend analyses would be inappropriate and of very little value. A 
special study that allowed the use of mean grades for trend analysis purposes 
was conAicted by ETS in the early 1980'8 for Lyle Jones, who published his 
findings in a widely distributed article in Science , entitled "Achievement 
Test Scores in Mathematics and Science." These results showed little average 
change in math or science achievement from 1973-1979. 

Scaled data from the equated portions of the math and scif^nce AP exams are not 
K^S^?f^L?ff?2?®^' so would require a project of longer duration 

SSsidirld fofthe'^fStSrl^^ contract. However, lucfi a project^could be 



TABLE 

Advanced Placement Mean Grades^ 1975-1984 



BE^*^ COPY AVAILABLE 



^^75 19 76 1977 1978 1979 1980 1981 1982 1983 1984 

Aaerican History 3.1 3.1 3.1 3.2 3.12 3.08 3.06 3.09 3.11 3.10 

History of Art 2^9 3.0 3.0 3.2 3.24 3.41 3.20 3.20 3.16 3.13 

^51222 . Li Id 3.4 3.4 3.33 3.31 3.35 3.31 3.30 3.25 

3J 3.1 3.1 3.1 3.09 3.03 3.02 3.01 3.05 3.02 

Engl. Co«p./Lit 3J 3.1 3.1 3.1 3.10 3.08 3.07 3.05 3.07 3.05 

Eur opean Hist. 3J 3.1 3.2 3.2 3.09 3.15 3.03 3.15 3.10 3.13 

French Language _3J 3.1 3.1 3.1 3.13 3.06 3.01 3.05 3.03 3.01 

German Literature 3.0 3.0 3.1 3.0 3.13 3.11 3.11 3.10 

Latin/Vergil 3.4 3.3 3.3 3.28 3.22 3.23 3.23 3.12 3.09 

rl^' ' M 14 3.0 3.03 3.02 3.06 3.08 3.09 3.13 



Calc. - BC 3.4 3.3 374 O 



3.28 3.19 3.32 3.33~ 09 OF 



Music rneory 3,^ 3,0 3.2 3.1 3.18 3.0 9 3.15 3.13 3.05 3.04 

C ine^h';^ H H 14 2.96 2.93 3.05 2.97 2.91 2.95 

^ (Mech.) 3.4 3.4 3.4 3.4 3^8 06 Ot 3~42 3~31 TTT 

C (Elec. . Mag .) 2.9 ..4 3.3 3.4 3.38 3.37 3:25 3;26 I'.ll " 1^41 

Spanish Language 3J 3.1 3.1 3.2 3.31 3.61 3.45 3.48 3.36 3.49 

SOU^: This table was prepared by Edicational Testing Service in September 1984 from annual charts (1975 through 1984) , 
entitled "Distribution of Candidate Grades - Advanced Placement Examinations." (The Advanced Placement Program 
is a program of the College Board.) 
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